A new manner of computer interaction is now in its infancy. The words "virtual environment" or "virtual reality" will soon be commonplace. A virtual environment is an environment where some portion of the environment is artificially simulated, most often via a computer. A computer may create a graphic simulation of an environment, complete with graphic images of chairs, windows, doors, walls, etc., and even images of other people. The computer may also simulate environmental sounds. The generated objects may be viewed on a common two-dimensional display, such as a computer screen, or, by viewing with special stereoscopic equipment, the objects may be made to appear three dimensional.
The most natural way for an individual to interact in a virtual environment is to directly control a graphical representation of himself For example, if the individual turns his head, the display screen at which he is looking is appropriately updated. Also, if the individual reaches out and closes his hand, the computer generated image of his hand on the screen reaches out and closes. Such virtual environments have been discussed in the literature.
To create the sensation of virtual reality, the computer should be able to generate and manipulate graphic images of real or imaginary objects in real time. Although generating a graphic representation of an environment may be time consuming and non-trivial to implement, much of the theory has been explored and is well-understood by those skilled in the art of interactive 3-D computer graphics and solid modeling. The invention described here pertains to the important related area in which relatively little research has been done, i.e., "How may a human user perceive grasping force and from his computer-generated counterpart in the virtual environment?"
There are many peripheral devices which have been created to allow a user to enter information into the computer. The most notable of these is the standard QWERTY keyboard. Besides the numerous modifications of this "key input" concept, there are many other devices with their associated permutations. A partial list of such devices includes mice, joy-sticks, trackballs and Computer-Aided-Design (CAD) tablets. The main drawback of these computer input devices is that they don't permit human users to enter information in a manner which may be the most efficient and natural. For example, in a CAD software program, the human designer may wish to rotate a 3-D graphic representation of a block on a computer screen to view and modify the hidden side. Using currently available input devices, the designer must select the axis or a sequence of axes about which the object must be rotated to achieve the desired orientation and view. After the desired axis is selected, the amount of angular rotation must be determined, usually by the linear motion of a mouse or by entering the desired amount of rotation as a decimal quantity via the keyboard. This whole procedure seems very awkward and non-intuitive when compared to what a person would normally do when confronted with a similar task in the "real world," i.e., he would simply reach out, pick up and rotate the object.
Instrumented gloves which provide finger-position information to the computer have been used to manipulate simulated objects in virtual environments. Such gloves have also been used in telerobotics to control highly dextrous end-effectors to grasp real objects. However, lack of force feedback to the glove wearer has reduced the effectiveness of these open-loop manipulation approaches. Imagine a 3-D graphic model of an egg on a computer screen. Suppose you are wearing a glove which maps your finger and hand motions to a graphic image of a hand on the same screen as the egg. As you move your hand and fingers, the corresponding graphic images of the hand and fingers move in a similar manner. The task is to move your own hand and fingers to control the graphic hand on the computer screen to pick up the egg. To accomplish this task you must provide enough force to reliably grasp and lift the virtual egg, but not so much force such that the egg is crushed. Without some kind of grasping force and tactile feedback, this task would be extremely difficult.
Attempts have been made to provide information about simulated contact with virtual or telemanipulated objects to senses other than the corresponding tactile senses. One method of simulated feedback which has been tested uses audible cues. For example, the computer may beep when contact is made. Another simple method is to highlight the object once contact is made. Both these methods will require the user to re-learn hand-eye coordination. It may be frustrating and time consuming for the user to learn one of these "unnatural" methods of grasping an object, and the sensation of interacting in a virtual environment will be reduced.
More recently, approaches have been developed to directly exert forces to the fingertips. One such approach uses pneumatic pistons located in the palm of the hand to exert resistive forces at the fingertips. The disadvantages of such an approach are numerous. First or all, pneumatic cylinders have low mechanical bandwidth and cannot exert very large forces because the limited workspace of the palm limits their size. Additionally, such actuators tend to be noisy and the fact that they are located in the palm limits the range of motion significantly. Other approaches have used servo-motors located directly on the back of the hand. Such approaches tend to be quite bulky and often need to be supported by robotic arms and thus are not well suited for desktop applications. When robotic arms are not used, hand and arm fatigue are often a problem as it is quite difficult to produce a device that is small and light enough for prolonged usage. Additionally, such devices often do not provide feedback to all the fingers in an effort to minimize bulk. Finally, such devices typically suffer from a limited range of motion which hinders manipulation.
Therefore, it will be appreciated that there remains a need for a man-machine interface for the hand that is capable of sensing finger and hand positions and hand orientation, that provides appropriate force-feedback, and that overcomes the other limitations in the state-of-the-art as described herein before.
One object of the invention is to provide a man-machine interface which may be employed in interactive computer applications. Another object of the invention is to provide a force feedback control system capable of controlling a set force to a selected part of the body, e.g., the fingertip et another object of the invention is to provide a man-machine interface comprising a glove capable of sensing finger and hand positions and hand orientation, which may exert, measure and dynamically vary and control the forces applied to each finger. Another object of the invention is to provide a digital control system capable of sensing the force applied to the fingertip and capable of using this applied force signal to control the fingertip force to a desired force set point which may vary as a function of finger position. Still another object of the invention is to provide a force feedback system which may be employed in many different applications, such as virtual environments, telemanipulation and interactive 3-D graphics, telerobotics and Computer Aided Design (CAD). Yet another object of the invention is to provide more natural and intuitive feedback during object/environment interaction.